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The number of computers in schools for instructional use has increased dramatically in 
recent years to nearly nine million nationwide (Anderson and Ronnkvist, 1999). The 
ratio of students to computers is around 5 to 1 . Internet connectivity continues to be a 
powerful force in education with technology being integrated more fully into the 
educational process (Market Data Retrieval, 1999). Though the statistics differ 
somewhat by grade, the vast majority (80-88 percent) of students report that they use 
computers at home or school to write short stories or papers and to learn things (U.S. 
Department of Education, 1998). Educators are using technology for instructional 
purposes including assessment and it is clear that the testing process can be enhanced 
through the use of technology (Sampson, 2000). 

Educators use assessment for a variety of reasons that range from individual student 
assessment to program evaluation to system-wide educational accountability (Wiggins, 
1993). Good assessment provides objective information that can assist in diagnosing 
problems and identifying curricular areas that need improvement (Anastasi, 1988). 
Assessments can help an individual find his or her way in the decision-making journey 
or help a school system travel the road to educational excellence. To navigate the 
journey accurately and safely, certain guidelines or rules of the road should be followed. 

This paper highlights the rules or best practices that should be followed by educators in 
evaluating, selecting, and using technology-delivered assessments. The guidelines 
adhere to existing professional standards such as the Standards for Educational and 
Psychological Testing (1999), Responsibilities of Users of Standardized Tests (1985) 
and the Code of Fair Testing Practices in Education (1988). The guidelines are 
presented in checklist form so that educators can use it to perform an initial evaluation 
as to whether a technology-delivered assessment is of sufficient quality that it can be 
accepted for a particular educational use. Adherence to these guidelines is of vital 
importance in reviewing, selecting, and using technology-delivered assessment 
instruments. 

TECHNOLOGY-DELIVERED ASSESSMENT 
CHECKLIST 



Guidelines for Test Quality 
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* The test content has been described by the developer, reviewed by the user, and 
matches the purpose of the testing. Assessment items are aligned with the desired 
areas intended for assessment. 



* Clear and supportable statements are provided by the test developer about what the 
test is intended to measure. 



* Evidence is provided by the test developer and has been reviewed by the test user to 
determine if the assessment is appropriate for the test taker with regard to age, 
membership in a subgroup, educational level, disability, and language competence. 



* There is ample evidence of comparability in the scores between the paper/pencil and 
technology delivered versions of the same test; where necessary, appropriate 
adjustments are made to scores to better assure comparability. 



* Evidence is provided to indicate the conditions under which the test results have been 
found reliable. The strength of that reliability has been reported by the test developers 
and examined by the individual who is considering using the test. 



* Validity evidence is provided by the test developer to substantiate suggested uses and 
interpretations. Test users apply and interpret the test results for purposes consistent 
with the validity evidence. 



* Limitations of the test and test results are clearly specified by the test developer and 
examined by the test users prior to selection of the testing instrument. No assessment is 
without limitations. 



* Published reviews of the assessments crafted by qualified persons have been 
reviewed prior to test selection and use. 
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* Test items and technical information have been examined to determine their currency. 
Careful examination of the age of the test and its technical information are considered 
before justifying the use of antiquated data or test versions. 

TEXT SCORE REPORTING AND 
INTERPRETATION 



* Tests are accurately scored and results have been returned as quickly as possible, 
together with test interpretations that are comprehensive and accurate, given the 
limitations of the test. 

* Assessment results are not over- or under-interpreted. Care has been taken not to 
develop interpretations or explanations that go beyond that which can be supported by 
the reliability and validity evidence. 

* Test reports and supporting manuals specify appropriate and inappropriate uses of the 
assessment results. 

* Reports include an indication of the degree of accuracy of the scores. 

* Score reports specify which interpretations are supported by research and which 
interpretations are based on professional opinion. Sufficient information has been 
offered to allow the test taker to weigh the credibility of the opinion. 

* Interpretations of test scores include the limitations of the test and test results, with 
reference to common misinterpretations of the assessment results. 

* If scores are used in high-stakes situations such as graduation, promotion, college 
entrance, placement, and credentialing, an explanation on how the passing or cutoff 
scores were set is available to the test user. 

* The score and interpretation report reveal where further information can be obtained 
about the test and score interpretation, and how a test taker can verify or challenge the 
accuracy of the score. 

* When interpreting a test score, relevant aspects of other information about the 
individual, such as previous test scores, educational level, and other performance 
indicators are used to augment the interpretation. This information is used to help the 
test taker gain further insight into the meaning of his/her test results. 

TEST ADMINISTRATION 
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* The test setting is comfortable, quiet, and conducive to allowing the test taker to 
maximize performance. The arrangement of the computers ensures privacy and 
comfort. 

* Testing equipment is in good working order and the software and/or Internet 
connectivity has been verified to be operating properly. 

* A site administrator is available to troubleshoot problems that may occur due to 
equipment, software, or other technology failures. 

* Policies and procedures have been established, explained to the test taker, and 
applied in cases of technology failure. For example, if there is a computer crash or a 
power disruption, are the responses to the test items saved, or does the test taker need 
to begin the task again? 

* Test takers are comfortable with the test format and use of keyboard or other 
equipment. If there is a question about the test takers' familiarity with the technology, 
practice exercises or tutorials have been provided to enable them to become facile with 
the equipment and the format, allowing the test taker to focus on the assessment rather 
than the mode of delivery. 

* Test items and answers are protected from compromise. Security of the equipment 
and test items is critical to the fairness to test takers of current and future test 
administrations. 

* The identity of the test taker is verified, particularly in high-stakes testing. 

* Tests are administered according to the procedures specified by the test developer, 
particularly in cases where standardization is important. 

* Both test users and test takers are informed about on whether individual score 
information is stored and, if so, where, and for how long. Periodic purges of individual 
test results stored locally or centrally may be advantageous in maintaining privacy. It 
may be more desirable for an individual to save test results on a personal disk rather 
than on a server or local computer. 

CREDIBILITY OF THE DEVELOPERS 



* Information is available for test users and test takers to determine the identity and 
professional credibility of the test developer. A determination has been made of 
adequacy of the test developer qualifications. This may be most important when 
assessments are published by small companies or by individuals without a relevant 
history of work quality. 

* Test developers provide information indicating whether they subscribe to precepts of 
the various testing standards produced by professional organizations. Professional test 
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developers who pledge adherence to various testing standards do attempt to produce 
high-quality assessments with sufficient technical and research support. 

ACCESS TO PROFESSIONALS 



* Because it is unlikely that the technology-delivered assessment programs will have 
information on the background and experiences of the test taker, a professional 
educator or counselor is available to provide value-added interpretations to the 
assessment information. 

* Human judgment is provided where assessment results suggest specific actions on 
the part of the test taker or the specification of interventions to modify a situation and 
will never be replaced by technology alone. 

STAYING CURRENT, KEEPING UPDATED 



The speed of change in the availability and use of technology in education continues to 
accelerate. Because of these changes, it is important for professionals to keep tabs on 
the latest developments in the assessment and technology areas. Information on web 
sites frequently is expanded and updated faster than information in print format. For 
these reasons, the following web sites should be checked periodically to identify new 
and important information related to assessment and the use of technology. 



* American Counseling Association (http://www.counseling.org) 



* American Educational Research Association (http://www.aera.net) 



* American Psychological Association in Science Directorate 
(http://www.apa.org/science) 



* American Speech Language Hearing Association (http://www.asha.org) 



Association for Assessment in Counseling (http://www.aac.ncat.edu) 



Page 6 of 9 



ED446327 2000-04-00 Technology-Delivered Assessment: Guidelines for Educators 

Traveling the Technology Highway. ERIC/CASS Digest. 



ERIC Resource Center 



www . eric . ed . gov 



* Association of Test Publishers(ATP) (http:/www.testpublishers.org) 



* Consortium for Equity in Standards and Testing (CTEST) 
(http://www.csteep.bc.edu/ctest) 



* Council for Exceptional Children, Directory of Current Projects in Assessment 
(http://www.cec.sped.org/osep/6assessm.htm) 



* Education Commission of the States (http://www.ecs.org) 



* ERIC Clearinghouse on Assessment and Evaluation (http://www.ericae.net) 



* ERIC Clearinghouse on Counseling and Student Services 
(http://www.uncg.edu/edu/ericass) * Joint Committee on Testing Practices 
(http://www.apa.org/science/jctpweb.html) 



* National Assessment of Educational Progress (http://www.nces.ed.gov/naep) 



* National Assessment of Educational Progress 
(http://www.ed.gov/pubs/ncesprograms/assessment/ surveys/naep.html) 



* National Association of School Psychologists (http://www.naspweb.org) 



National Association of Test Directors (http://www.natd.org) 
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* National Center on Educational Outcomes (http://www.coled.wmn.edu/nceo) 



* National Center for Education Statistics (http://www.nces.ed.gov/index.html) 



* National Center for Research on Evaluation, Standards, and Student Testing 
(CRESST) (http://cresst96.cse.ucla.edu/index.htm) 



* National Council for Measurement in Education (http://www.ncme.org) 

THE JOURNEY CONTINUES 



Technology provides enabling tools for educators and counselors in locating information 
on assessment, organizing and maintaining test information, building relationships 
between and among test results and counseling and guidance interventions, using 
alternative assessment techniques, assessing difficult areas and higher order skills, 
introducing efficiencies into the assessment situation, and reaching audiences that are 
not normally accessible without the availability of certain technologies. Clearly 
technology, used properly and creatively, can advance opportunities for individuals and 
guide them through a journey where they can flourish. 
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